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Gapless surface states that are protected by time reversal symmetry and charge conservation 
are among the manifestations of 3D topological insulators. In this work we study how electron- 
electron interaction may lead to spontaneous breaking of time reversal symmetry on surfaces of such 
insulators. We find that a critical interaction strength exists, above which the surface is unstable 
to spontaneous formation of magnetization, and study the dependence of this critical interaction 
strength on temperature and chemical potential. 

PACS numbers: 



Topological states of matter are one of the main themes 
of modern condensed matter physics. A recent important 
discovery in this field are Topological Insulators |2J. 
In three dimensions Topological Insulators are electronic 
materials that have a bulk gap like an ordinary insulator, 
but have conducting states on their surface i3j . The 
gapless spectrum on the surface arises from the nontrivial 
topological order of the band structure [4[ |5] . It is com- 
posed of an odd number of Dirac cones in the Brillouin 
zone of the two dimensional surface. These Dirac cones 
cannot be gapped for as long as time reversal symmetry 
and charge conservation are maintained. 

In this work we explore the conditions under which the 
surface of a strong topological insulator may lower its 
energy by a spontaneous breaking of time reversal sym- 
metry through the formation of a uniform spin polariza- 
tion. The way energy may be saved by spin polarization 
is most easily understood for the case where there is one 
Dirac cone on each surface, and the chemical potential 
lies at the Dirac point. For such a case, two sources for 
energy gain may be identified: first, if the spin polariza- 
tion introduces a mass term to the Dirac Hamiltonian, 
the filled electronic states are lowered in energy, while 
the energy of the unfilled states is raised. And second 
- as usual for electrons - if all spins are polarized, two 
electrons cannot reside at the same point, and the short- 
distance energy cost is reduced. This is not the case in 
the absence of magnetization. Even though the half filled 
Dirac cone includes just one electron for every momen- 
tum state, the variation of the spin's direction with that 
of the momentum leads to a non-zero probability of two 
electrons being at the same point, and thus to a gain of 
interaction exchange energy by spin polarization. 

In our analysis, carried out within the Hartree-Fock 
approximation, we find that there is a critical interac- 
tion strength above which the spin polarized ground state 
is favorable in energy compared to the non-interacting 
ground state. We find this to be the case for every 
value of the chemical potential, for all temperatures much 
smaller than the bulk energy gap, and for both contact in- 
teraction and Coulomb interaction. Of the possible spin 
polarizations we find that the lowest energy corresponds 
to an out-of-plane polarization, which introduces an en- 
ergy gap to the Dirac cone at the Dirac point. For contact 
interaction, we find the phase transition from the unpo- 



larized to the polarized phase to be of second order both 
as a function of interaction strength and as a function 
of temperature. Finally, we find the energy gap to rise 
sharply as temperature is lowered below the temperature 
Tc of the phase transition, and reach values comparable 
to Tc at about T ^ 0.9Tc. We note that a gapped Dirac 
cone was recently observed by angle resolved photo emis- 
sion (ARPES) in the topological insulator phase [10], 
but the information available is not sufficient to judge 
whether this observation is related to the mechanism we 
study. 

We start with the non-interacting Hamiltonian. 
ARPES measurements carried out on two topological in- 
sulators, Bi2Te3 and Bi2Se3, have found that while near 
the Dirac point the spectrum is Dirac-like, an hexago- 
nal warping of the Fermi surface occurs away from the 
Dirac point [6*, T . This phenomenon was modeled by Fu 
[8 , who suggested an effective Bloch Hamiltonian of the 
surface including a cubic term: 

1-L{k) = VQ(kx(Jy - kyCFx) + \k^cos(?>0)cFz (1) 

where vq is the electron velocity near the Dirac point, A 
is the warping parameter and <j are the Pauli matrices. 
The above spectrum contains two branches that meet 
at the Dirac point. This Hamiltonian is time-reversal 
symmetric. At the time-reversal invariant point A: = the 
spectrum is degenerate and the degeneracy is protected 
by the time-reversal symmetry. 

The interaction part of the Hamiltonian is 

nx = J drdr'ijl{r)i;^{r)V{r-r^)iplir^)ip^^ (2) 

Here V^J(r) is the creation operator of an electron at point 
r and spin a. We will assume that the following operator 
has a non vanishing expectation value: 

=M^,^.(r-r') (3) 

Strictly speaking, we can interpret M as the magnetiza- 
tion only when r = r'. However, the expectation value 
(|3| breaks time reversal symmetry even when r ^ r' . 
We confine ourselves to uniform states, in which M will 
depend only on = r — r^ This case is most relevant 
when the Fermi-energy is located at the vicinity of the 
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Dirac point, otherwise spin- wave formation is an inter- 
esting possibility. 

The expectation value ([3| may be written in a matrix 
form as M(j^a'ir-r') = Mo{R) ■ I + M{R) ■ a, but since we 
are interested only in the spin part of the interaction and 
in particular the out-of-plane spin {z direction), we will 
assume that only ^ 0, hence: Mcr^a'ir-r') = M{r)Gz' 
In-plane magnetization will be discussed in a later part 
of the paper. 

By denoting = J dRViR)M{R) exjp {-ik ■ R), we 
can write the mean-field Hamiltonian as: 
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where 
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dRV{R)M^{R) ■ 
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dk 
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where dk = kdkdO and /cp is the Fermi-momentum, which 
is the solution to vok^ + A/cp = /i, with fi the chemical 
potential. In general the integration has an upper cutoff 
in momentum, /ca, which corresponds to the momentum 
at which the surface states merge with the three dimen- 
sional bands. For A = all results depend linearly on this 
cutoff, but for A 7^ we can identify a momentum scale 

1/2 

qq = above which the integrand decays as k~^. 

The cutoff dependence then disappears for /ca >> qq. 

We find that there is a critical interaction strength gc 
at which a second order phase transition occurs, such 
that for g > Qc there is an m 7^ solution to ^ with 
< 0. The critical interaction strength is 



y{pa) - y{pf) 1 _ 



h = vo{kxCry — kyCFx) + {\k^ cosW — 2Fk)cFz 

For a self-consistent determination of the out-of-plane 
magnetization, we denote the eigenvalues and the corre- 
sponding eigenstates of h as e±(k) and ( Xt^X; )± re- the function Y is defined by: 
spectively. The ± signs refer to the positive and negative 
energy states. Hence, the magnetization per unit volume 
is: 



,(/i = 0) 



(8) 



where Pa, F = ^a,fa/ — is a dimensionless parameter and 



Y(p) 



p'dp' 



(27r)2 7o 



de- 
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where 
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(p'2+ (p'3cos36')^) 



3/2 
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The function Y{p\) is a continuous, positive and mono- 
tonically increasing function of with the following 
asymptotic behavior: 



and fe is the Fermi-Dirac function at energy e. The trivial 
solution (m = 0) always exists, but we are interested in 
non trivial solutions of this equation. The total energy 
per unit volume is: 

Em= [ dRV{R)M\R) + ^ ^(e-/e_ + e+/ej (6) 

k 

We will define the energy cost to have finite magnetiza- 
tion as = Em — Eq. If we find a non trivial solu- 
tion to Eq. ([5| that minimizes Em (gives AE < 0), we 
will conclude that the system forms spontaneous surface 
magnetization. 

Although it is hard to determine the exact validity con- 
ditions for this variational procedure and to estimate its 
accuracy, it is still possible to define a necessary condi- 
tion. Since the procedure above relies on the existence 
of a bulk gap, it is clear that it requires the interaction 
energy to be smaller than the bulk gap. We will consider 
the condition Uc < Ai^uik as necessary for the above pro- 
cedure (where Uc is the critical interaction energy and 
Abuik is the bulk energy gap). 

It is simplest to analyze a contact interaction, V{R) = 
At zero temperature Eq. ^ for the magnetization 
M(o) = m becomes: 



Y{pa) 



,PA«l 

0.247 ,pA » 1 



(10) 



becomes gc = 27r - ^ 



For /i = and for A ^ Eq. ^^v.^v^.l^xv.o yc — 

while for /ca >> (x)^^^ ^ becomes ^ 4.1 • ^/v^. 
The resulting phase diagram (/i = 0) in the large cutoff 
limit is presented in Fig. ([T]). 

Clearly, for all values of A and /ca > /^f, the critical 
interaction strength gc increases as increases. In the 
limit A ^ Eq. ([8| gets a simple form: 



9c{^) = 9c{^ = 0) 
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k\ 



m 



ikp 



dk 



gm — Xk^ cos 3^ 



^^^^^ J v^k^ ^ {gm- Xk^ cos 30y 



(7) 



As expected for k^ k\ (full band) magnetization for- 
mation is not possible. We note, however, that as long 
as the chemical potential is far from the bulk bands its 
effect on gc is rather weak. 

For g ^ gc can estimate the interaction energy per 
particle by noticing that (Hx)per-particie = where Uc 
is the electron density on the surface. We will use ARPES 
data of Bi2Te3 and Bi2Se3 surfaces [71|9| to compare the 
critical interaction energy with the bulk energy gap for 
a typical surface density (table |l|. In both materials 
Uc < Abuik^ which puts the mean field results within the 
necessary condition for the validity of the approximation. 

At non-zero temperatures the transition happens when 
the free energy of the magnetized surface is lower than 
that of the unmagnetized one. 
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TABLE I: Bulk gap vs. Uc for rie = 5 • lO^^cm" 



BisTes BisSes 
/^huik 0.16 eV 0.35 eV 
Uc 0.078 eV 0.066 eV 



It is useful to note that: (cr^-) = —{cf^j^) = (cr^). For 
fj^ = 0^ the equation for m becomes: 



m 



-I 



dk x(m, k) • tanh 



(27r)2 



(11) 



where T is the temperature and x(m,k) = — 
AA:^cos36>. The free energy difference (A/) per unit vol- 



1 + cosh (A/t;gfc2 + xl^/T) 
■ cosh (v/wp^ + (AA;3 cos 30")2/y) 



(12) 



It is natural to define the dimensionless temperature scale 

r = where = (^)^^^ is the characteristic energy 
scale introduced by hexagonal warping. The resulting 
phase diagram is presented in Fig. ([T]). 

As expected, the critical interaction strength gc in- 
creases with increasing temperature. We note however 
that the effect on gc is rather mild. Again, the phase 
transition is of second order, both for a fixed tempera- 
ture as a function of ^, and for a fixed ^ as a function of 
temperature. 

A similar analysis may be carried out to examine the 
conditions for in-plane magnetization to be formed. To 
that end, we assume a magnetization in the x — y plane, 
solve the self consistent equation that determines its mag- 
nitude and find the conditions under which it leads to 
an energy gain. As we now show, for typical parame- 
ters (vo^X^kA) the critical interaction strength to have 
in-plane magnetization gc,x-y is smaller than ^c,^? the 
critical interaction strength to have a magnetization per- 
pendicular to the plane. We carry out the calculation at 




FIG. 1: Phase diagram (pA — 2) for contact interaction at 
zero (dashed line) and finite temperatures, where r = 
A critical interaction exists, beyond which magnetization for- 
mation occurs. The critical interaction increases as the tem- 
perature increases. 



zero temperature. We note that an in-plane magnetiza- 
tion does not create a gap in the spectrum. It does how- 
ever break time-reversal symmetry and hence removes 
the topological protection of the Dirac-point. 
The Bloch Hamiltonian is: 



1-L{k) = vo{kxCry — kyCJx) + A/c^ cos ?>Ogz — grn • a 



(13) 



We denote {rrix^my 



mil. In the symmetry broken 



phase, the system will choose a specific direction. Al- 
though the system is not rotationally invariant we find 
that the angle dependence is very small, and we choose 
^11 = ^a;- The self consistent equation for rrix is then 
given by: 



where 



+ '^o^sin^ 



(14) 



\Jv^k^ + (grrix)'^ + 2gvomxk sin + {Xk^ cos SOy 



All that is left now is to solve Eq. (14), minimize the 
energy and deduce the critical interaction gc^x-y The re- 
sulting gc,x-y has the same form as Eq. ([8|, with Y{pa) 
being replaced by ^(pa), where ^(pa) is a continuous, 
positive and monotonically increasing function of pa with 
the following asymptotic behavior: 



^(pa) 



^ .PA « 1 



0.352 ,pa >> 1 



(15) 



The ratio 



= ^l^^j is presented in Fig. (j2j) for /i = 
the instability to appears before 



9c, z 

0. For ^A < 3.7 

the instability to in-plane magnetization. Since this value 
of kA corresponds a warping parameter that is about a 
hundred times larger than that observed in Bi2Te3, we 
conclude that the interesting quantity to examine is the 
magnetization in the z direction. This conclusion does 
not change when the chemical potential is away from the 
Dirac point. 




FIG. 2: The ratio between the critical interactions to have 
in-plane/out-of-plane magnetization as a function of the ratio 
of kA to the warping wave vector scale (x)^^^- For typical 
values the instability to niz occurs before the one to nix- 
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FIG. 3: Phase diagram for Coulomb interaction at zero tem- 
perature. A critical interaction exists, beyond which magne- 
tization formation occurs. 



Our analysis has so far been focused on contact inter- 
action between the electrons, and has uncovered an in- 
stability of the surface to the formation of magnetization, 
with the instability being strongest for the formation of 
out-of-plane magnetization at zero temperature and zero 
chemical potential. The dependence of the critical inter- 
action strength on temperature and chemical potential is 
however rather weak. 

Motivated by these observations we turn to the long 

2 

range Coulomb interaction, V{R) = ^, and study the 
instability to the formation of out-of-plane magnetization 
at zero temperature and zero chemical potential [11] . We 
assume that (?/^J.(r)?/^cr'(^')) = M{R)az is an exponentially 
decaying function of R with a decaying length a~^. This 
assumption yields two self consistent equations for the 
magnetization and for a: 



m 



r dk 
J (2^ 



47rme 



Xk^ COS 30 



/ 27rme 
(2^)' Jy2u2^(^ 



Xk'^ COS 3(9 



27rme^ 



Afc3 COS 36*)' 



(16) 



(17) 



The trivial solution, in which m = and a is arbitrary, 
is always an exact solution to the equations above. After 
solving numerically for m and a and minimizing the en- 
ergy, we identify the regions of parameter space in which 
a non-trivial solution exists, as shown in Fig. (|3|. The 
values we obtain for range between 1 and 4 lattice 



constants, hence justifying the interpretation of m as the 
magnetization. 

For the Coulomb interaction the mean interaction en- 
ergy per particle is of the order of e^r^^, where tq is 

the mean distance between electrons on the surface. We 

_ 1 

can estimate tq ~ ng ^ . For a given density we can 

parameterize the critical interaction strength by defin- 

1 

ing a critical interaction energy Uc = e^r^ ~ e^rie . 
Again, for our approximation to be valid we must have 
Uc < ^buik- We find this condition to be satisfied for 
both Bi2Te3 and Bi2Se3: Using Fig. |3] and extracting 
the density from ARPES data we evaluated the critical 
interaction strength, ^ 0.057eg (0.072eg) for Bi2Te3 
(Bi2Se3), where eo is the electron charge in vacuum. We 
find the actual Coulomb interaction on the surfaces of 
these two materials to be too weak for an instability. 
The dielectric constant on the two surfaces is close to 
40, as compared to a value of about 15 for which our 
approximation leads to an instability. 

To summarize, we examined here the possibility that 
time-reversal symmetry is spontaneously broken on the 
surface of a three dimensional strong topological insula- 
tor due to interactions between surface electrons. We as- 
sumed that the surface can be described by a 2D effective 
Dirac Hamiltonian, and treated interactions within the 
Hartree-Fock approximation. We found that for a strong 
enough interaction, both of the contact and the Coulomb 
types, the surface is unstable to the formation of sponta- 
neous magnetization, with the strongest instability being 
to the formation of magnetization in the direction per- 
pendicular to the surface. For the contact interaction, 
we found the transition from the unmagnetized to the 
magnetized surface to be of second order, both at zero 
temperature as a function of interaction strength and for 
a fixed interaction strength as a function of temperature. 
The dependence of the critical interaction strength on 
temperature and chemical potential, at least for the con- 
tact interaction, is rather mild. We found the interaction 
strength in the two most studied strong topological in- 
sulators to be too weak for an instability, but not by a 
large factor, making the instability an issue that may be 
relevant for other topological insulators. 
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